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ABSTRACT 

The Boston, Massachusetts, school district requires, that its 
9th grade students pass both the Boston Public Schools Math Benchmark 
Assessment (BPS Math) and the Scholastic Reading Inventory (SRI) before 
entering 10th grade. At Madison Park Technical-Vocational High School in June 
2000, 349 students failed either the mathematics or reading test or both, and 
approximately half of them attended a 5-week Summer Transition Program. Jobs 
for Youth-Boston ( JFY) performed a computerized instructional component of 
the Summer Transition Program using the PLATO system. An evaluation of PLATO 
use was conducted. In actual practice, most students received 40 to 50 
minutes of PLATO computer-based instruction daily, using the PLATO versions 
available in 2000. In this time, the average student mastered 8.7 PLATO math 
modules and 7.6 reading modules. Average gain scores from pretests to 
posttests were one standard deviation higher for the BPS Math and two 
standard deviations higher for the SRI. In fact, 42% of the summer school 
students achieved high enough math posttest scores, and 70% received high 
enough reading scores, to qualify for entry into the 10th grade. Higher score 
gains in mathematics were only weakly correlated with the mastery of more 
mathematics modules. For reading, higher score gain was significantly 
correlated with more time spent studying PLATO modules. PLATO was well-liked 
by teachers and students. Some recommendations for the improvement of PLATO 
implementation are made. An appendix contains four supplementary tables. 
(Contains 17 figures and 1 reference.) (Author/SLD) 
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Executive Summary 



The Boston school district requires that its ninth grade students pass both the Boston 
Public Schools Math Benchmark Assessment (BPS Math) and the Scholastic Reading 
Inventory (SRI) before entering tenth grade. If ninth graders do not pass both tests, they 
are required to attend summer school and then repeat the failed test or tests in August. 
They must pass both tests in order to go on to 10* grade. At Madison Park Technical- 
Vocational High School in June 2000, 349 students failed either the math or reading test 
or both; approximately half of them attended the five-week Summer Transition Program. 
For this remedial program, ten BPS teachers gave classroom instruction. Jobs For 
Youth — Boston (JFY), an independent private non-profit agency working with the 
Boston Public Schools, conducted a computerized instructional component of the 
Summer Transition Program using the PLATO system. "Madison Park was the lowest- 
scoring school in the state on the MCAS [the Massachusetts Comprehensive Assessment 
System — the statewide achievement test] and these were its lowest-achieving students," 
said Gary Kaplan, executive director of JFY. 

Program Description. Four days a week for five weeks students attended two hours 
for math, two hours for reading, or four hours for both, depending upon what test(s) they 
had failed. Their two-hour subject blocks were divided between classroom instruction 
and PLATO computer instruction, for a maximum of twenty hours total computer use 
time throughout the program. In actual practice, excluding days for placement and final 
tests, most students received 40 to 50 minutes of PLATO computer-based instruction 
daily. Students were assisted in the labs by their classroom teacher and one to three 
support personnel. They used the older versions of PLATO English and reading, which 
were available in summer 2000. 

Student Achievement. Students spent from two to nineteen hours studying math 
and from one to ten hours studying reading on PLATO. In this time the average student 
mastered 8.7 PLATO math modules and 7.6 reading modules. This PLATO study in 
combination with classroom instruction helped to produce large gains on the math and 
reading re-tests. 

Gain scores were computed from the pre-program to post-program BPS Math 
Assessment and SRI. Average gain scores from pre-tests to posttests were one standard 
deviation higher for the BPS Math and two standard deviations higher for the SRI. Stated 
in other terms, forty-two percent of the summer school students achieved high enough 
math post-test scores — and seventy percent high enough reading scores — to qualify for 
entry into the 10th grade. 

PLATO Use and Achievement. Math students who did better on the first BPS 
Math Benchmark test tended to master more math modules; students who did less well on 
the test mastered fewer modules. However, higher score gains — demonstrating 
improvement between tests — only weakly correlated with more math modules mastered. 
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Reading students were similar in regard to test scores. Those who got higher pre- and 
post-test reading scores also mastered more modules. For reading, higher score gains — 
demonstrating improvement between tests — was significantly correlated with more time 
spent studying PLATO modules. 

Other Lessons. Additional lessons learned or confirmed about PLATO were as 
follows: 

• PLATO was well liked by teachers and students. 

• Hardware capability — speed and memory — has a strong bearing on the ease of 
use of PLATO software. 

• There is a learning curve for teachers and technical staff; they need more than a 
few hours of training to deal with hardware and software and to make best use of 
the instructional possibilities. 

• Labs ran best when there were at least two staff members present to handle 
technical problems, give instructional help, and monitor student behavior. 

• The Help Desk is appreciated. 

• PLATO Assessments are very long. 

• In the reading program there were problems with screens freezing or looping; the 
frequency of such problems could not be determined. The revised English 
modules (available after this program) are much appreciated; users like color and 
graphics. 

• For drills and mastery tests, the similarity of items in each item pool is noticeable 
and may cause boredom for students and encourage creative approaches by 
teachers in tutoring — which may or may not produce learning outcomes or be 
time efficient. 



Contact Information 

Gary M. Kaplan, Executive Director 
Jobs For Youth - Boston, Inc. 

125 Tremont Street 
Boston, MA 02108 
(617) 338-0815, ext 224 
GKaplan@JFYAte/.org 
www.jfyboston.org 
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Jobs For Youth-Boston 

Madison Park Technical-Vocational High School 
Boston, Massachusetts 



Introduction 



In the Commonwealth of Massachusetts, high school students are required to pass a 
tenth-grade examination called the Massachusetts Comprehensive Assessment System 
(MCAS) in order to graduate from high school. The Boston Public Schools also have 
their own benchmark tests at various grade levels. All ninth graders must pass the Boston 
Public Schools Math Benchmark Assessment (BPS Math) and the Scholastic Reading 
Inventory (SRI). Passing scores in June 2000 were 70% for BPS Math and a raw score of 
43, or 975 lexiles, for the SRI. These two tests serve as pre-tests for the 10 th grade 
MCAS. Students who do not pass them must attend a remedial summer school program 
and then pass a re-test in order to go on to the 10 th grade. If they do not pass the tests, 
they must repeat the ninth grade. 

At Madison Park Technical-Vocational High School in June 2000, 349 students failed 
one or both of the tests. About 185 chose to attend the summer school session that ran for 
five weeks, from July 10 to August 12. After administering PLATO assessment tests, 

145 students were identified for math remediation and 68 for reading. Both math and 
reading students were given an hour of classroom instruction by BPS teachers and an 
hour in a computer lab using PLATO. 

The lab portion was directed by Jobs For Youth - Boston (JFY), a private, non-profit 
agency that provides various education and job training programs for youth and adults. 

For the past few years, JFY has been helping schools and community agencies set up 
MCAS tutoring programs using PLATO. JFY provided the PLATO software and support 
people for the computer labs; they also trained the classroom teachers in the use of 
PLATO. PLATO was used for skill assessment and development, with the curriculum 
aimed directly at the math and reading tests. 

Program Description 

The School. Madison Park Technical-Vocational High School is a monolithic 1970's- 
era structure that covers two city blocks in the inner city of Boston. With an enrollment 
exceeding 1600, it is the largest high school in the city. As a technical-vocational high 
school, it gives its students both academic classes and vocational training during alternate 
weeks of the school year. Therefore, Madison Park students receive half the math and 
language arts instruction they would in a non-vocational high school. Madison Park’s 
2000 MCAS scores were the lowest in the city and in the state. The 10 Ih grade failure rate 
in math was 97% and in English 88%. 
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The school houses several computer labs that are networked and connected to the central 
BPS system. Two labs were available to program students for their first two weeks; 
thereafter, three and sometimes four labs were in use. Each lab had approximately 20 
computers; together, the three most commonly used labs had 70 computers. 

The Students. Madison Park students were likely to have many challenges in getting 
an education. Gary Kaplan, executive director of JFY, pointed out: "Madison Park was 
the lowest-scoring school in the state on the MCAS, and these were its lowest achieving 
students." Furthermore, many students were bilingual or non-English speaking. 
Alternative languages were primarily Spanish and Haitian Creole. Some proportion of 
these students no doubt had learning disabilities and attention problems, and many were 
known to have problematic home lives. While behavior and attitude were not part of this 
study, student motivation and compliance throughout the city’s summer school program 
were not high. 

While at least 349 Madison Park students had failed the BPS Math exam and/or SRI 
reading exam, not all of them attended or remained in the summer school remedial 
program. In addition, for those who did attend, not all of the data necessary for 
evaluating the program were available. For purposes of this analysis, student data were 
used for each topic when three pieces of information were available: a pre-test score, a 
post-test score, and PLATO use information. The resulting number of learners studied 
for this report was 77 for math (53% of the 145 identified math students) and 47 for 
reading (69% of the 68 identified English students). Students who attended the program 
were given the PLATO Assessment so that the outcomes would further inform the design 
of the learning pathway in both subjects. 

The Staff and Their Computer Experience. Ten teachers were provided for 
the program by the Boston school district to offer instruction in math and reading. They 
were recruited by district-wide postings of the positions and were hired according to 
seniority; most did not have experience using computers in education. IFY staff said, 
"One teacher had computer skills and experience; two others 'caught on fast’." Eight of 
the teachers were present for the one day of PLATO training given by IFY. During that 
day, teachers took the PLATO assessment test as part of their training and were thereby 
shown the skills students would be taught. 

IFY had been using computers for instruction for one year at the time of the program 
being examined in this report. Their software of choice was and still is PLATO. Gary 
Kaplan, who made the selection of PLATO, says that it is “the best thing on the market 
right now for comprehensive remediation and skill building.” IFY added PLATO to the 
Madison Park computer labs for this summer program, taking special care in connecting 
to the central Boston Public Schools network, as they had been requested to do. 

PLATO labs were directed by JFY's loan Reissman, a twenty -year veteran teacher from 
Texas. loan has had ten years of experience in using computers in her classroom; in 
addition, as a result of three days of original PLATO training and five years of 
experience, she is very conversant with the software. Suzanne Rickard, the PLATO 
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curriculum and instruction consultant, also helped in the labs, particularly at the 
beginning of the program, setting up the curriculum that students would use and a student 
incentive program for success. Even at the end of the program she was still there about 
three days a week, helping with computers and students. One of two JFY computer 
technicians — George Criss or Paul Lapsley — was usually available to help with 
troubleshooting. George said, "At first we called the [PLATO] Help Station frequently 
with problems. Now we are top-notch PLATO people — we don't need the Help Station 
much." JFY also provided a counselor, Joe Thompson, to manage student discipline. 

Program Design. The unifying purpose of the Madison Park summer school 
program was to enable students to pass their second attempt at the BPS Math and SRI. 
The program employed PLATO as a primary instructional piece for skill development 
and PLATO assessments were based on test goals. The program also included classroom 
time. Teachers were told to plan their own classroom instruction and to likewise cover 
the topics on the BPS Math and SRI. 

This special summer school program was held four hours each day, from 8 a.m. to 12 
noon, four days a week (Monday through Thursday) for five weeks, for a total of twenty 
days. Students were enrolled for two hours of math instruction daily, or two hours of 
reading instruction, or both, depending upon their math and reading test scores. In a two- 
pronged approach, students received approximately one hour of classroom instruction 
and one hour of PLATO instruction a day, with the BPS classroom teacher participating 
in both. (Half of the teachers started with the lab hour, and the other half started in their 
classroom; after an hour they switched places.) It was planned that students would 
receive a total of 20 hours in the classroom and 20 hours on PLATO for each subject. 

While it was intended that students use the PLATO program for a total of 20 hours, in 
actual practice the time was less. The first three or four days of the program, depending 
upon which class students were in, were spent scheduling students, assigning computers, 
and giving PLATO assessments; the last two or three days were spent on post-testing. 
Absenteeism was high, although students were technically not allowed to miss more than 
two days. Occasionally, teachers did not bring their classes to the lab. On a daily basis, 
computer time was less than an hour, since some students spent several minutes moving 
between classrooms and labs; some had a long walk in the large facility, while others 
took a break on the way. And some did not pay good attention. Actual maximum time 
on task in the labs working with PLATO instruction was about 40 to 50 minutes each day 
for 14 days. This totaled to 9 to 12 hours on task for math or reading instruction in 
PLATO. 

Some students logged more time in math than would be expected. The PLATO use 
records indicate that overall more time on task was spent on math than on reading. From 
interviews it was learned that some students did math during their reading lab time, in 
part because some students just liked to do math (and indeed many needed math skills) 
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and in part because the math program was more engaging than the reading instruction 
was at that time 1 . 

In cases where students worked in pairs or in larger groups, the PLATO use statistics 
would only be recorded for the student who logged onto the workstation; time for the 
other users would not be recorded by the system. The Advanced Reading Strategies 
course was subject to technical problems, with machines “crashing” with some regularity. 
At those times, students worked in pairs or threes since machines were limited. In 
addition, some students worked with partners because of their limited English ability. 

Student Placement in PLATO. The week before the program started Suzanne 
Rickard met with Joan Reissman and two program teachers to design the PLATO 
assessment for math and reading. They worked to match the BPS Math and SRI test 
goals as much as possible (therefore also matching the eventual MCAS requirements at 
the same time). The resulting instruction that students received from modules the 
assessment said they needed was maximally targeted to the tests, with the hope that as 
many students as possible would pass at the end of the summer. As Suzanne said, "This 
test was make it or break it for these kids for the tenth grade MCAS." The resulting 
PLATO objectives were aligned to 65% of BPS Math and 90% of the SRI. Suzanne used 
the PLATO Custom Assessment Tool (PC AT) to prepare the assessment test; she did not 
use FASTRACK or Work Keys Locator. 

During the first two weeks the program was provided with only two computer labs, rather 
than the three that had been arranged. Joe Thompson, a Madison Park counselor 
working for JFY, coordinated schedules so that a class was tested in each lab for all 
available hours. By the end of the first week 120 students were assessed for math and 30 
were assessed for reading. The following week testing proceeded as much as possible. A 
few classes could not use the labs until the third week of the program. 

As the program progressed, it became clear that some students lacked the pre-requisites 
to succeed at the necessary modules. In response to this need, Suzanne and Joan 
constructed a set of remedial modules for kids who needed more background information 
or skills, primarily in math. In addition, they also added more modules in reading for 
those students who completed the program faster than anticipated. 

Student Computer Use. Most students worked alone on computers. Those who 
worked in groups of two or three did so when there was a lack of equipment (particularly 
the first two weeks or when the reading crashed), when they wanted a change of activity, 
or when they needed help with English. One student did not speak any English, so all of 
his instruction was received in groups. Another, who was bilingual and very good at 
English, gave a lot of help to others and therefore did not log as much computer time as 
she might have. 



1 As noted previously, a new reading curriculum has been released since the time covered by this 
evaluation. — ed. 
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On a typical day, students worked quietly in the lab, raising their hands when they had 
questions or asking a friend for help. One, two or three students might be working with 
other students. In two of the labs, many students worked in pairs because of language 
problems. Not all teachers came to the lab at the beginning, but interest and involvement 
increased so that by the end teachers were usually there with their classes. Joan 
Reissman was there to assist teachers, help with computer problems and tutor students as 
much as possible, moving between the three or four labs at any given time. One of the 
technology support staff, either George or Paul, was usually there. Joe Thompson, a 
veteran BPS counselor and student services co-ordinator, was there every day to handle 
discipline problems. 
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PLATO Use and Outcome Data 



The Boston Public Schools established a promotion policy, effective spring 2000, which 
sets high expectations for all students so they will have the knowledge and skills needed 
to succeed at each grade level. The policy sets grade-by-grade requirements that learners 
must complete to be promoted to the next grade. For the 1999-2000 school year, 9th 
grade students were promoted to grade 10 if they met three requirements: 

1 . Receive passing grades in specified classes in English language arts, 
mathematics, and other academic areas. 

2. Receive a passing score of 70% correct on the Boston Public Schools Math 
Benchmark Assessment (or a passing score on another specified assessment). 

3. Receive a passing score of 975 lexiles or higher on the Scholastic Reading 
Inventory (or a passing score on another specified assessment). 

Students in grade 9 who do not pass the math and/or reading test receive a failing grade 
in the subject(s) for the year. This grade can be changed to a passing grade only if the 
students attend the Summer Transition Program and pass the reading and/or math test by 
the end of the program. The students participating in this study were drawn from the 
population of Summer Transition Program attendees for the summer of 2000. 

Complete data were available for 77 math learners and 47 reading learners. Analyzing 
these data makes it possible to examine how growth in student achievement is related to 
PLATO useage. The findings for math learners are presented first followed by findings 
for reading learners. 

Math Curriculum 

PLATO Math System Use. Students were enrolled in PLATO math modules as 
part of the Summer Transition Program. The average math learner mastered 8.7 modules 
in PLATO mathematics. The number of modules mastered ranged from one to 48 wiith 
the middle half of the learners mastering between 4 and 1 1 modules. The average student 
spent 10.7 hours studying PLATO math modules. For the 77 learners with complete 
data, time spent on PLATO ranged from a low of about 2 hours to a high of about 19 
hours. Table 1 below and Tables A1 and A2 in the Appendix provide descriptive 
statistics on PLATO use by math learners. Figure 1 shows the distribution of learner time 
across math modules studied. 



Table 1. Math Descriptive Statistics 





N 


Minimum 


Maximum 


Mean 


Std. 

Deviation 


# Modules Mastered 


77 


1.0 


48.0 


8.7 


7.7 


Hours on PLATO 


77 


2.1 


19.2 


10.7 


4.2 


BPS Math, June 


77 


6.0 


68.5 


32.3 


14.8 


BPS Math, August 


77 


10.0 


90.0 


51.4 


22.1 


Pre-Post Gain 


77 


-36.5 


70.0 


19.1 


22.9 
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30 % 



25% 




0to3hrs 5 to 7 9 to 11 13 to 15 17 and up 

3 to 5 7 to 9 11 to 13 15 to 17 

Time on PLATO Math 

Figure 1. Distribution of Learners' Time on PLATO Math Modules 

Math Achievement Gains. The BPS Math Benchmark Assessment taken in June 
2000 served as the pre-test for analyzing how much students gained by participating in 
the program. The mean pre-test score was 32.3 with a standard deviation of 14.8. The 
range of pre-test scores was from 6.0 to 68.5. Math learners were post-tested with the 
same BPS math test in August 2000 after mastering one or more PLATO math modules. 
The average posttest score was 5 1 .4 with a standard deviation of 22. 1 . The range of 
posttest scores ran from 10.0 to 90.0. Of the 77 math students in this study, 32 (42%) 
achieved a passing score of 70% or higher on the post-test. Considering how far below 
the passing point many of these students started this was a substantial achievement. 

Subtracting the pre-test score from the post-test score yields an average gain of 19.1 
points. One common way of judging how meaningful such a gain score might be is to 
use the average standard deviation of the pre- and post-scores as a measuring stick. Doing 
this we find that the gain of 19.1 points is about one standard deviation in size (the 
average of the two standard deviations is 18.5). This magnitude of difference is usually 
considered large and educationally meaningful, particularly when looking at changes 
occurring within a relatively short period of time as in this study. Table 1 provides 
descriptive statistics for the pre- and post-BPS math scores. Figure 2 shows the mean 
percent correct on the pre- and post-test BPS math scores. 
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Figure 2. Mean Percent Correct on Math Pre- and Post-Tests 



Math Correlations. One of the key purposes for the evaluation is to examine the 
relationship between the level of use of the PLATO system and changes in student 
achievement. This was addressed by looking at the correlation between the number of 
modules mastered, hours of PLATO use, and learner achievement scores. 

Results indicate that there is a positive correlation between the number of math modules 
mastered and pre- and post- BPS math achievement scores. This indicates that higher test 
scores were positively related to completing PLATO modules. A weak positive 
correlation was also observed between the number of modules mastered and the BPS 
math gain score. Figures 3 and 4 present scatterplots of the data for number of modules 
mastered and achievement scores. These figures also include a best-fit regression line to 
predict achievement scores given the number of PLATO modules mastered. No 
significant relationships were found between the hours of PLATO use and BPS math 
achievement scores. 



Table 2, Correlations Between PLATO Math Variables (n = 77) 





# Modules 
Mastered 


Hours on 
PLATO 


BPS math 
June 


BPS math 
August 


Pre-Post 

Gain 


# Modules Mastered 


1.00 


0.32* ""I;: 


0.26* life 


<m*' • 


! 0.19 


Hours on PLATO 




1.00 


0.04 


0.13 


0.1 


BPS math June 






1.00 


0.28*- 1 5 •;.! 


-0.38* 


BPS math August 








1.00 


i:.0T9€I'l 


Pre-Post Gain 










1.00 



* Correlation significant at .05 level 
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■X* Observed 
Score 

Predicted 

Score 



Number of PLATO modules mastered 

Figure 3. Scatterplot of Post-BPS Math Scores and PLATO Math Modules Mastered 




Observed 

Gain 

Predicted 

Gain 



Number of P LATO modules mastered 

Figure 4. Scatterplot of BPS Math Gain Scores and PLATO Math Modules Mastered 

The relationship between modules mastered and BPS math gain scores is further 
illustrated in Figure 5. This figure shows the average gain score for each of four equal- 
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sized groups. The first thing that should be noted is that all four groups have relatively 
large gains over the pre-test: 9 to 24 point gains. In addition, the trend is for the gain 
scores to be larger as the number of modules mastered increases. 

It should be noted that all such findings must be tempered by considerations of how valid 
and reliable the measures are. As shown in Table 2, the test-retest correlation between 
the pre- and posttest scores was quite low (r = 0.28). Low test-retest correlation may be 
caused by selecting only a restricted range of students to test — such as testing only the 
lower-achieving students in a school — which was the situation here. Whatever the reason, 
low test-retest correlations means that all other correlations, such as with PLATO system 
use, will likely be lower than what they could be if test-retest correlation were higher. 

This suggests that the underlying relationships between PLATO use and learner 
achievement may be stronger than indicated by these findings. (See Glass and Hopkins, 
1984 for a discussion of how restricting the range of students tested can reduce the 
observed correlation between two measures. They also provide a means to estimate the 
full correlation if a number of assumptions are met. Unfortunately, these assumptions, 
such as large sample size and a bivariate normal distribution, do not appear to be met in 
this case.) 




1to3 4 to 5 6 to 10 11 and up 

Students Grouped by Modules Mastered 

Figure 5. Distribution of Average BPS Math Gain Scores by PLATO Modules Mastered 
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Reading Curriculum 



PLATO Reading System Use. The 47 reading learners in this study were recorded 
as mastering fewer PLATO modules than did the math learners. The average reading 
learner mastered 7.6 modules in PLATO reading. The range of modules mastered went 
from three modules to 1 7 modules, with the middle half of the learners mastering 
between six and eight modules. The average student spent 4.5 hours studying PLATO 
reading modules. For the reading learners this ranged from a little over one hour to a 
little over ten hours. Table 3 and Tables A3 and A4 in the Appendix provide descriptive 
statistics about PLATO use by reading learners. Figure 6 shows the distribution of 
learner time across reading modules studied. It should be noted how many of the learners 
had PLATO reading time in the lower end of the distribution. 



Table 3. Reading Descriptive Statistics 





N 


Minimum 


Maximum 


Mean 


Std. 

Deviation 


# Modules Mastered 


47 


3.0 


17.0 


7.6 


2.8 


Hours on PLATO 


47 


1.2 


10.3 


4.5 


2.2 


Reading June 


47 


415.0 


1050.0 


704.1 


133.6 


Reading August 


47 


640.0 


1170.0 


970.0 


128.1 


Pre-Post Gain 


47 


-5.0 


595.0 


265.9 


125.3 




Time on PLATO Reading 

Figure 6. Distribution of Learners' Time on PLATO Reading Modules 
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Reading Achievement Gains. In June 2000, the reading learners completed the 
SRI Level HS-Alt2 Form A reading pre-test. The mean learner pre-test score was 704. 1 
Lexiles with a standard deviation of 133.6. The range of pre-test scores was from 450 to 
1050 (one learner had a pre-test score above 975). Reading learners were post-tested on 
the SRI Level HS-Alt2 Form B in August 2000 after completing one or more PLATO 
reading modules. The average post-test score was 970.0 Lexiles with a standard 
deviation of 128.1. The range of post-test scores ran from 640 to 1170. Of the 47 learners 
studied, 33 (70%) received post-test reading scores of 975 lexiles or higher, and therefore 
qualified for entry into the 10th grade. 

Subtracting the pre-test score from the post-test score yields an average reading gain of 
265.9 Lexile points. Using the average standard deviation as a reference, we find that 
this gain is about two standard deviations in size. This magnitude of difference is usually 
considered very large and educationally important. The use of alternate forms of the SRI 
test helps guard against increases in scores just due to familiarity with particular test 
items. Table 3 provides descriptive statistics for the pre- and posttest reading scores. 
Figure 7 shows the mean pre- and post-test SRI lexile scores. 



1000 




Reading Pre Reading Post 



Figure 7. Mean Reading Pre- and Post-Test SRI Lexile Scores 

Reading Correlations. As with the math scores, we examined the relationship 
between the level of PLATO use and changes in student reading achievement. This was 
addressed by looking at the correlation between the number of modules mastered, hours 
of PLATO use, and learner achievement scores. 
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There was a positive correlation observed between the number of reading modules 
mastered and pre- and post-reading achievement scores. As with the math scores, this 
indicates that higher reading scores were positively related to completing PLATO reading 
modules. However, for reading there was no significant correlation observed between the 
number of modules mastered and the reading gain score. This may be because there was 
much less variability in modules completed; 51 percent of the reading learners mastered 
exactly eight modules. 



Table 4, Correlations Between PLATO Reading Variables (n = 47) 



# Modules 
Mastered 


Hours on 
PLATO 


Reading 

June 


Reading 

August 


Pre-Post 

Gain 


# Modules Mastered 1.00 


p;48*ll: : !; 




i 0.30* =I=:4 


I -0.03 


Hours on PLATO 


1.00 


-0.02 


0.35* ill 


m%* ■■■ 


Reading June 




1.00 


o,54*: : . 


-0.51*. 


Reading August 






1.00 


MmM, 


Pre-Post Gain 








1.00 


* Correlation significant at .05 level 



On the other hand, there was a significant positive correlation between hours of PLATO 
use and reading gain scores (r = .38). Figures 8 and 9 present scatterplots of the data for 
number of hours using PLATO and reading achievement scores. These figures also 
include a best-fit regression line to predict achievement scores given the number of hours 
using PLATO. 
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Figure 8. Scatterplot of Post-Reading Scores and PLATO Reading Hours 
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Observed 

Gain 

Expected 

Gain 



Hours on PLATO 

Figure 9. Scatterplot of Reading Gain Scores and PLATO Reading Hours 

The relationship between time on PLATO and reading gain scores is illustrated further by 
dividing the reading learners into four equal-sized groups and plotting the average gain 
for each group. As with the math learners, it should first be noted that all four reading 
groups have large to very large gains over the pre-test: 183 to 320 lexile gains. In 
addition, the trend is strongly for the gain scores to be larger as the number of modules 
mastered increases. 
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Figure 10. Distribution of Average Reading Gain Scores by PLATO Hours Completed 
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PLATO Instructor Ratings 



Teacher surveys were administered initially in June 2000; only a few teachers responded. 
A follow-up survey was given in August 2001 to get a higher response rate from the 
participating teachers. Findings for the initial survey are given; then for the follow-up 
survey. 



Initial Teacher Survey 

Of the ten teachers at Madison Park High School, four answered PLATO questionnaires 
at the end of the JFY program in June 2000. Most of them were very positive about the 
PLATO program and their experience in the JFY program. The survey items were 
grouped according to five categories for this report: (1) the content of the instruction; (2) 
the instructional design of the program; (3) the screen design of what is presented; (4) 
teacher training; and (5) teacher affect about PLATO and computers. In evaluating their 
responses it should be remembered that neutral responses could indicate lack of 
experience, ambivalence, or unwillingness to make a judgment. 

Across all of the items, the pattern of responses pointed to high satisfaction with PLATO 
in general. Negative indicators dealt with software problems [probably with the reading 
course] and lack of adequate teacher training for the program. 

Content. Three of the teachers strongly agreed, and the fourth agreed, that the content 
was generally up-to-date and that it included what students needed to learn. The 
remaining content questions elicited slightly less enthusiastic but nevertheless positive 
replies. One teacher strongly agreed and three agreed that both the PLATO content and 
course objectives aligned with the course objectives and content that they used 
themselves. All four of the teachers agreed that there was adequate depth in the exercises 
and tests and that students generally understood the explanations. When asked about 
content being free of errors and inaccuracies, one teacher strongly agreed, one agreed, 
and two said that they were neutral on the issue. (Items 5, 1,2, 3, 8, 7, 4) 

Instructional Design. All four teachers strongly agreed that the quality and style of 
instruction was consistent throughout the program. They also strongly agreed that 
tutorials involved students with frequent questions, answers, and feedback. One teacher 
strongly agreed and two agreed that the module parts corresponded to the objectives in 
the Instructor Guides; one reply was neutral, possibly meaning that that instructor was not 
familiar enough with the Guides or the program to make a judgment. Overall, teachers 
were less positive or certain about students seeming confused or trapped by the system. 
Two of them agreed that their students rarely seemed confused or trapped by the system; 
two said that they were neutral on that issue. (Items 6, 10, 9, 19) 

Screen Design. All of the teachers agreed-three strongly— that color was used 
appropriately in the program. In the remaining four judgment items, three teachers were 
positive and one was neutral. Two teachers strongly agreed and one agreed that screens 
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were consistently readable, graphics were used appropriately, all courseware used 
consistent strokes and display style, and that the software was free of bugs or errors. 
(Items 13, 11, 12, 14, 15) 

Teacher Training/Ability. Computer abilities and training seemed to be the weak 
area of the teacher ratings. When asked whether they could use the student progress 
reports to identify students needing their attention, three agreed and one disagreed. To 
the statement “I was able to make appropriate individual student assignments on the 
system,” one strongly agreed, one agreed, and two were neutral. Training was rated 
lowest of all items, with only one respondent agreeing that training for PLATO was 
adequate; one teacher was neutral and two disagreed (no one strongly disagreed). All of 
the teachers agreed (one strongly) that they would like more training on how to use 
PLATO to best advantage in their teaching, an issue which goes beyond hardware or 
software. (Items 16, 24, 18, 25) 

T eacher Affect. All of the teachers agreed (two strongly) that they were able to spend 
time in one-on-one tutoring and counseling while students used PLATO. This seems to 
indicate that the labs ran reasonably well, which could be interpreted as giving teachers a 
positive experience. Three of the instructors agreed — two strongly — that they find 
working with the computer is generally a productive, rather than frustrating experience 
and that they enjoy working with the PLATO computer system. Three of the teachers 
said that the PLATO system played a useful role in their teaching (one felt strongly). 
Likewise, three teachers agreed (one strongly) that their students responded well to the 
PLATO system. One respondent gave a neutral response for all four of these items. 
(Items 17, 21,22, 23,20) 

Open-Ended Comments on Initial Survey. Teachers were asked five open- 
ended questions about PLATO use. Teachers were generally positive about the learning 
potential of the PLATO instruction. The questions and the answers provided by the four 
teachers who completed the initial survey are transcribed here verbatim: 

“7. What do you like best about teaching with the PLATO computer? ” 

• PLATO provides lots of examples and step by step instructions to solve problems. 

• Lessons are well outlined. 

• Provides additional reinforcement to topics taught in class. 

• Students develop skills. 

“2. What do you like least about teaching with the PLATO computer? ” 

• Doesn’t get the students to write 2 . 

• The time in the computer (1 hr and VT) is too long 3 . 

• The tutorials require the student to read. The teacher has to read for the student 
who has difficulty reading. 4 



2 The alignment used in the study did not include Math Problem Solving, which includes optional writing 
activities. It also did not include the writing curriculum. — ed. 

3 PLATO lessons are designed to be completed in 15-30 minutes each. -ed. 
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“3. How would you change the PLATO lessons? ” 

• Get students to write more. 

• More people to help low level students. 

"4. What suggestions do you have to improve the way you use the PLATO system? ’’ 

• I would like to have PLATO in the classroom to complement the textbook 
lessons. Also, more training and planning time to adequately use PLATO. 

• No suggestions. I’m still learning the system. 



“5. What other comments or suggestions do you have on the PLATO system or this 
course? ” 

• It is great for learning. I enjoyed working with it and all my students’ grades have 
improved. When I asked them what the reason for improvement was, they all said 
it was because of working with PLATO. 

• PLATO with all the activities enriches and extends the students’ understanding of 
the lesson by giving them the opportunity to explore content in greater depth. 



4 Most lower-level PLATO lessons do use “read the screen audio.” Future development plans include 
addition of audio to additional courseware, -ed. 
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PLATO Instructor Ratings (n»4) 



13. Color was used appropriately. 

5. Content was generally up-to-date. 

1 . The PLATO course content includes my students need to learn 
about topics taught. 

17. I was able to spend time in one-on-one tutoring and counseling 
while students used PLATO. 

10. Tutorials involved the students through frequent questions, 
answers and feedback, rather than just reading. 

6. Quality and style of instruction was consistent throughout the 
curriculum. 

22. I enjoy working with the PLATO computer system. 

21 . I find working with the computer is generally a productive, rather 
than frustrating experience. 

15. Screens were consistently readable. 
14. Graphics were used appropriately. 
12. All courseware used consistent strokes and display style. 

1 1 . Software was generally free of bugs or errors. 

25. I would like more training on how to use PLATO to best 
advantage in my teaching. 

3. The PLATO course content corresponds to the content of the 
stand end-of-course we test. 

2. The PLATO course objectives corresponded to those for my 
course. 

23. The PLATO system was a useful role in my teaching. 

20. My students respond well to the PLATO system. 

9. Tests, application/drill lessons, and tutorials corresponded to the 
objectives in the Instructor Guides. 

8. There was adequate depth of exercises and tests. 

7. Students generally understood the explanations. 

18. I was able to make appropriate individual student assignments on 
the system. 

4. Content seemed generally free of errors and inaccuracies. 

16. I was able to use student progress reports to identify students 
needing my attention. 

19. My students rarely seemed confused or trapped by the system. 

24. I was adequately trained to operate the PLATO system. 




Number of Instructors 



[□Strongly Agree C3Agree DNeutral □Disagree B Strongly Disagree 

Figure 1 1. Frequency of Teacher Ratings by Survey Item, June 2000 
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Mean PLATO Instructor Ratings (n=4) 



13. Color was used appropriately. 

5. Content was generally up-to-date. 

1. The PLATO course content includes my students need to learn 
about topics taught. 

17. I was able to spend time in one-on-one tutoring and counseling 
while students used PLATO. 

10. Tutorials involved the students through frequent questions, 
answers and feedback, rather than just reading. 

6. Quality and style of instruction was consistent throughout the 
curriculum. 

22. I enjoy working with the PLATO computer system. 

21 . I find working with the computer is generally a productive, rather 
than frustrating experience. 

1 5. Screens were consistently readable. 
14. Graphics were used appropriately. 
12. All courseware used consistent strokes and display style. 

1 1 . Software was generally free of bugs or errors. 

25. I would like more training on how to use PLATO to best 
advantage in my teaching. 

3. The PLATO course content corresponds to the content of the 
stand end-of-course we test. 

2. The PLATO course objectives corresponded to those for my 
course. 

23. The PLATO system was a useful role in my teaching. 

20. My students respond well to the PLATO system. 

9. Tests, application/drill lessons, and tutorials corresponded to the 
objectives in the Instructor Guides. 

8. There was adequate depth of exercises and tests. 

7. Students generally understood the explanations. 

1 8. I was able to make appropriate individual student assignments on 
the system. 

4. Content seemed generally free of errors and inaccuracies. 

16. I was able to use student progress reports to identify students 
needing my attention. 

19. My students rarely seemed confused or trapped by the system. 

24. I was adequately trained to operate the PLATO system. 
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Figure 12. Mean Teacher Ratings per Survey Item, June 2000 
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Follow-Up Teacher Survey 



The teachers were asked to complete questionnaires again about the PLATO system one 
year after they had participated in the Madison Park program; seven surveys were 
returned. The follow-up survey was done to obtain responses from more of the teachers 
who participated in the program. It also provided the opportunity to ask some additional 
questions not included in the original teacher survey. 

Responses for this August 2001 survey showed that instructors were very pleased with 
PLATO and said in various ways that the system worked. On a scale of 1 to 5, with 5 
being high, the topmost average for an item was 4.6; the bottom-most average was 3.6, 
which is also a high rating. Most of the items received one “neutral” response (or more), 
with different people giving that rating. With many of the items a “neutral” rating could 
indicate indifference, uncertainty, lack of experience, or both positive and negative 
feelings. Two of the seven educators said that the software had bugs and two said they 
did not receive enough training. Figures 13 and 14 display the survey answers for the 
August 2001 survey administration. 

Frequency of Teacher Instruction Regarding PLATO. The follow-up 
survey also asked teachers to rate how often they provided students information or 
instruction about the use of the PLATO system. As Figure 15 depicts, almost all of the 
instructors prepared their students for PLATO work daily or almost daily. They spoke to 
the students about the skills, knowledge, and attitudes that they needed to succeed at 
PLATO and helped them to relate lessons to personal experiences. Five of the seven 
respondents said that they explained specific PLATO objectives students would meet, 
and six explained how the skills and knowledge learned in PLATO would fit into the 
overall course or lesson goals. All seven said that they reminded students of the reward 
system for completing PLATO, and all but one said they told students how they could get 
help if they needed it. 
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PLATO Instructor Ratings (n=7) 



24. Computer work is productive, not frustrating. 
21. In general, students respond well to PLATO. 

17. I could do tutoring while students used PLATO. 
1. PLATO content for these topics is good. 
26. The PLATO system was useful in teaching. 
25. I enjoy working with the PLATO computer system. 
23. My students respond well to the PLATO system. 

4. Content seemed generally free of errors. 

1 8. I could make student assignments on the system. 
28. I would like more training on using PLATO well. 
8. There was adequate depth in exercises and tests. 

5. Content was generally up-to-date. 

6. Quality, style of instruction consistent throughout. 
16. I was able to use student progress reports. 
14. Graphics were used appropriately. 
1 3. Color was used appropriately. 
12. PLATO used consistent keystrokes, display style. 
10. Tutorials involved students through interactive style. 
9. All lesson parts aligned with Instructor Guides. 

7. Students generally understood the explanations. 
22. Students are seldom confused, trapped by PLATO. 
3. PLATO content aligns with our standard final test. 
2. PLATO course objectives aligned to my own. 
15. Screens were consistently readable. 
20. I was able to relate PLATO to classroom activities. 

19. Students had enough time on PLATO. 
27. I was adequately trained to use PLATO. 
1 1 . Software was generally free of bugs or errors. 




0 1 2 3 4 5 6 

Number of Instructors 



Figure 13. Frequency of Teacher Ratings by Survey Item, August 2001 
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Mean PLATO Instructor Ratings (n=7) 
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14. Graphics were used appropriately. 

13. Color was used appropriately, j 
12. PLATO used consistent keystrokes, display style. 

10. Tutorials involved students through interactive style. 

9. All lesson parts aligned with Instructor Guides. 

7. Students generally understood the explanations. 

22. Students are seldom confused, trapped by PLATO. • j 
3. PLATO content aligns with our standard final test. £ 
2. PLATO course objectives aligned to my own. r 
15. Screens were consistently readable. 

20. I was able to relate PLATO to classroom activities. r 
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27. I was adequately trained to use PLATO. 

1 1 . Software was generally free of bugs or errors. 
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Figure 14. Mean Teacher Ratings per Survey Item, August 2001 
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Frequency of Teacher Instructions Regarding PLATO Assignments (n=7) 



Clearly identified to the students 
the rewards and incentives for 
trying hard and doing well within 
the PLATO system. 



Explained to the students specific 
procedures for getting support if 
they didn’t understand something 
they were trying to learn within the 
PLATO system. 



Helped the students relate what 
they were about to leam in their 
PLATO assignments to their cxvn 
personal previous experiences. 



Explained to the students how the 
skills and knowledge learned 
within their assigned PLATO 
modules fit into the overall course 
or lesson goals. 



Articulated to the student(s) in 
some way those prerequisite 
skills, knowledge, or attitudes 
needed to fully succeed with their 
newly assigned PLATO modules. 



Described to the students the 
specific objectives they were 
going to learn within their 
assigned PLATO courses or 
modules. 




0 1 2 3 4 5 6 7 



Number of Teachers 



| □ Each Session HMost Sessions □ Occasionally □ Never [ 

Figure 15. Frequency of Teacher Instruction Regarding PLATO, August 2001 



Note: The scale points “At the beginning of each semester or marking period ” and “Maybe one 
time during the year " were not selected by any teachers and were therefore not depicted in 
Figure 15. The one instructor who marked 0, “Never", considered PLATO an assessment tool 
only and took the class twice a week to the lab. 
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Open-Ended Teacher Questions. The follow-up survey also asked seven open- 
ended questions for teachers to answer. Two new questions were added in addition to the 
initial open-ended questions on the June 2000 survey. The questions and the answers 
provided by the seven teachers who completed the August 2001 survey are transcribed 
here verbatim: 

1. What do you like best about teaching with the PLATO computer? 

• PLATO is flexible; it allows students with different learning styles equal 
opportunity to learn, master the skills presented. It provides individual learning 
experience, encouraging student- teacher interaction in finding answers. 

• Students seem to learn mathematics much faster and less stressfully. I especially 
liked the constant and immediate feedback students received. 

• The PLATO program is beneficial to students at any grade level. I have seen 
some of my students make tremendous result achievements in two weeks by 
attending a PLATO computer session every day. 

• It gave the students the opportunity to work on problems alone and be tutored in 
the process. 

• Self-paced instruction to meet individual needs. 

• The amount of examples given. 

2. What do you like least about teaching with the PLATO computer? 

• I liked everything about it. 

• Not enough training. 

• Sometimes it is difficult to supervise the work of students, especially when 
Internet or games are available on the system. 

• Sometimes the computer was looking for an answer to be presented one way only, 
and if it was presented another way the student got it wrong. 

3. Was there a regular time within the sequence of a lesson or unit in which your students 
experienced their PLATO modules? For example, did they visit the computer lab during 
the introduction to a new lesson or unit? Or did they use PLATO during the activity or 
information phase of a lesson? Or during the practice, review, or transfer (application) 
phase? Or was PLATO used only as an assessment tool? If yes, briefly explain. 

• All students were introduced to the PLATO system on the computer for the 
lessons, practice, review and assessment also. 

• After introducing a new lesson, students will visit lab and use PLATO for practice 
or assessment tool. 

• Our students visited the computer lab every other day. This schedule worked as a 
review for the lesson from the day before. 

• We used PLATO as an assessment tool; we went twice a week. 

• PLATO was used for my students at that particular time as an introduction to a 
different assessment tool. 

• Students were assigned specific classroom time for PLATO; modules not based 
on need. 
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4. Describe any strategies you employed to determine whether or not the PLATO modules 
assigned to each student were the most appropriate for ensuring their success in your 
class. 

• Knowing my students' abilities, I was able to talk to the PLATO administrators 
about putting together modified activities suitable for each one’s needs. 

• There are many ways for students to grow, many ways to leam. By identifying 
individual learners, PLATO was the most appropriate means to ensure their 
success during the summer transition. 

• They were given short practice tests from the PLATO system curriculum. 

• Pre-testing / post-testing, degree of difficulty. 

• We used PLATO during 8 to 10 days — not enough time to answer these 
questions. 

5. How would you change the PLATO lessons? 

• Add more activities for special education students. 

• Develop basic concepts suitable for students with special needs. 

• I cannot offer any changes in PLATO lessons. 

• I wouldn't change anything. 

• Not sure — don't remember. 

6. What suggestions do you have to improve the way you use the PLATO system? 

• Teachers need more time to leam to use the PLATO system than two days. 

• Connecting the mathematical ideas to the real world by the different real-life 
applications. 

• More frequent scheduling of PLATO modules. 

• I will need more time to evaluate or make any suggestions on how to improve the 
PLATO system. My school does not use the PLATO program. It was only a 
summer tryout. 

7. What other comments or suggestions do you have on the PLATO system or this 
course? 

• I was amazed at how easily some of my special education students were able to 
leam the math skills on the computer, particularly basic algebra skills. 

• I would like to see the PLATO system being available to our students at Madison 
Park for the whole school year and not just for the summer. 

• More training. At times the bugs in the computer system were difficult to 
diagnose. 
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Educator Interviews 



Student Work in Computer Labs. Throughout the interviews, JFY staff members 
said they were very happy with the PLATO topics taught and with the system's strategy 
for instructing and involving students. 

Teachers wanted their students to go through the tutorial, and many of them wanted 
students to do the drill as well. Instead, when they could, many students went right for 
the mastery test; when they were bumped out they did the tutorial. (Passing the mastery 
test required four out of five correct answers.) One JFY staff member said that students 
could get out of a tutorial in two minutes and try the mastery test again; Suzanne pointed 
out that users must get to a certain point in the concepts before they can get out. 

Just as PLATO recommends, in the Madison Park program PLATO modules and courses 
were sometimes modified for individual students as they went through the courses. For 
math, if a teacher made a judgment that a student needed basic concepts, PLATO 
modules were added to provide it. "They were backed down as needed," Joan reported. 
She also put them into pre-requisite or parallel modules in order to avoid repetition when 
students had failed a mastery test once or twice. She said that problems in the tutorial 
were repetitious, since they come from one item pool, and instead of running students 
through one drill many times, she chose to put students into similar modules for 
additional instruction and practice. Finally, some students were given additional modules 
when they had finished the prescribed modules, primarily in reading. In reading, 
classroom teachers came and explained the material and assigned students to modules 
where they needed to be. 

Occasionally, a teacher would ask if certain modules could be presented on a certain day. 
They would say, “I did this topic today; could students do it on PLATO today also?” 

That was not possible because of the large numbers of teachers and students and the 
relentless pace of the daily schedule. JFY staff commented that it would be easy to do in 
a normal school setting without the breakneck pace of the summer program. 

Student misbehavior in the computer labs was infrequent. As George Criss, a JFY 
computer technician, said, "Kids can download a Napster file, rename PLATO, and 
change the screens. Students at Madison were about the same as at any school. We have 
projects at other schools and those students change the screens and so on. We can go in 
with lock down security, and then when you leave the project you have to go in and 
remove it. It takes a lot of time. Now that we are comfortable with PLATO we can do 
more [to prevent student misuse].” 

Encouragement for Student Work. Everyone who was interviewed about 
PLATO expressed the sentiment articulated by one educator: "Students were very 
engaged and got a lot out of it." Students were motivated by both the computers and the 
high-stakes nature of their work. External controls also helped: there was little 
distraction, since Internet access had been removed and there were no games; also, 
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students were required to stay in their seats. "Kids who had attitude eventually stopped 
coming," said Suzanne. 

External rewards were also added to strengthen student motivaton. Suzanne set up a 
token system in which students earned stars for mastering modules. Students liked the 
stars and kept track of what they garnered. For a given number of modules they got a 
multi-colored pen or a movie ticket. Rewards were funded by JFY and PLATO. 

Instructional Support for PLATO. Any preparatory or whole-class instruction for 
PLATO work was handled in the classrooms by teachers. Since the classroom instruction 
was not managed by JFY, teachers determined the correspondence between classroom 
instruction and PLATO content. Teachers became more involved in the PLATO time as 
the program proceeded, until all of them regularly attended with their classes and helped 
individual students with whatever they needed. Joan, Suzanne and other occasional staff 
helped students in each lab with questions posed by hardware or software and did some 
tutoring as time allowed. 

Use of Practice Sheets. The lab supervisor said that PLATO practice papers were 
used in the lab a little bit, adding the caveat, "The computers were primarily supposed to 
be a break from the classroom. PLATO practice exercises aren't that great... I use my own 
things — I've taught 20 years and have so many resources. I have not been interested in 
curriculum, as such, but I do have these resources." 

Monitoring Student Progress. Students were not monitored while they were on 
the computers, except as teachers walked around the room and looked at what they were 
doing. Many teachers wanted students to do the tutorials, and some wanted them to do 
the drills as well. 

Teachers gave feedback to lab staff about individual students when they thought students 
might need changes in their program. Joan, the lab supervisor, ran reports for classroom 
teachers about once a week after the program was underway. She did not think the 
teachers used the information in their instruction; however, she was sure that they did use 
the reports to determine who had completed the number of units necessary to win a 
movie pass. 

Grading Student Work. For this summer program, students were given grades for 
their classroom work, but PLATO performance was not part of those grades. 

Gary Kaplan tells this story: "One of the least promising students, who got 14% on the 
math test in June, sat down at the computer the first day and figured out how to use 
PLATO in about twenty seconds. I watched him clicking away on the keyboard. He was 
doing Pythagorean Theorem, area of cones and cylinders, electromechanics. After about 
half an hour, I asked him how he liked it. He said, ‘This is cool, man. It’s a better way of 
learning.’ He sat there every day clicking away; we had to chase him out of the lab. He 
completed 19 modules. At the end of the summer his score was 75%.” 
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Program Implementation Problems. JFY felt that students did not have enough 
time on the computers, partly because there were not enough computers for the number 
of students the first two weeks; later, shortages occurred when machines were down and 
there were no extras for the displaced students who had been using them. There were 
also days lost to assessment at the beginning and testing at the end of the short program. 
Assessment should have been done ahead of time, one person said, so that students could 
start with instruction the first day in the lab. Likewise, student schedules should have 
been prepared ahead so that students knew what they were doing from the beginning. As 
it was, schedules were not prepared — the scheduler provided by the school had quit the 
night before the start of the program. 

The installation of PLATO required special attention since the district and school were 
concerned about their computer network being accessed by an “outside” entity. 

Assessing PLATO. Educators were asked to indicate the most and least liked 
features of PLATO. The following lists summarize their comments. 

Most liked features of PLATO: 

• It was individualized; kids controlled the pace. 

• The audio feature was good for the ESL students. 

• Math was graphically good. 

• Students felt they were getting personal attention. 

• Students could get around in the program; in fact, they could get around too 
well — some of them quickly figured out that they could spend two minutes in the 
tutorial and go right to the mastery test. 

• George liked the support of the Help Desk. "PLATO people are super- responsive 
and helpful." 

Least liked features of PLATO: 

• The assessment is too long. 

• Modules were done in a set order; some teachers would have preferred PLATO 
work to follow up on classroom presentations. 5 

• Joan is of the opinion that repeating Drills and Mastery Tests is not useful. She 
said that while "different mastery tests are supposed to be available, they are 
actually similar because they draw from the same item pool. Kids don't want to 
do the drill again. For problem solving, math sentences are good. Some PLATO 
is more drill and kill rather than thought provoking — it does do a good job of 
what it is." Rather than put struggling students who had failed mastery tests 
through repeated drills, she routed them through other modules that were pre- 
requisites or parallel topics. [This is what PLATO recommends.] 

• Reading/English was not good that year (summer 2000); there was too much gray 
[-colored screen] in it. “The new program is better.” 



5 The system can be set to work either way, with modules in any sequence. — ed. 
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• The Advanced Reading Strategies (ARS) program kept crashing or looping. "I 
was not sure whether it was a hardware or software problem." George referred to 
trouble with screens freezing and losing the student record of work at that point. 
He said, "I think it happened when students went from one module to another and 
then back again — when they bounced around. They lost what they had done for 
that day. Whenever you have to go back in you lose the data to that point." 

• Suzanne reported: “For the majority of the program, PLATO labs did not have 
major system problems. When students started advancing into their PLATO 
reading assignment, the [ARS program] had higher order graphics that needed to 
be adjusted in the workstation display setting. A black box was appearing in 
some of the screens that prevented the student from doing his or her work in that 
program. That situation was remedied by fixing the display and hardware video 
performance.” During the third week the computers started freezing up. Suzanne 
thought that perhaps this happened because of the additional demand of more 
students using it. 

• The PLATO assessment was not all that they needed that year. Joan said, “The 
Reading Program did not match the test goals. The new Reading Program would 
have matched it well, but it was not released until April of the following year. It 
wasn’t available to us.” 

Ways the Program Could Improve Its Use of PLATO. 

• "Relate it to the total instructional system; use it as reinforcement." It is good to 
include the computer with a teacher, but a teacher is necessary for some students. 
A variety of methods should be used to accommodate various learning styles. 

• Perhaps teachers should not make corrections or repairs. "A lot of teachers 
wanted to be involved with the computers. Maybe they tampered with programs. 
Maybe that is an issue. [Problems] would have been more easily resolved if we 
[were the only ones who] did repairs." 

• Better hardware makes the program easier to use. "There were more issues with 
hardware than with software. You can have the best software in the world, but if 
the hardware is not adequate, or the staff can't use it, or if untrained people tamper 
with it — you will have problems," George warned. 

Suggestions for Improving PLATO. One teacher commented that “Text should 
be more sensitive to the urban environment and the vocabulary of it. In one math lesson 
students were asked how many items were in an urn; it should have said bottle or jar. 

ESL students didn't get that word.” [They were aware of the built-in PLATO dictionary.] 

General Comments on the Use of Computers in Education. JFY offers 
programs that combine teacher instruction with computer instruction. Joan said, 
"Computers can get boring; you can't just stick [students] in front of a machine. They 
need some human interaction, if only for encouragement." She suggested that teachers 
interpolate some classroom instruction, perhaps as a 15-minute break in the lab time, to 
vary the experience for the students. "Break it up with a lecture. ..and speed things up." 
She went on to say that "computers can handle the main burden." 
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Gary Kaplan is convinced of the positive value of computers in education. "The question 
shouldn't be 'Why use computers?' but rather 'Why shouldn ’t teachers use computers?’ 
Every other profession does.” 

He continues, "When you talk to kids in the group they say this helps them learn. For 
kids who have attention problems, the classroom is a difficult environment. There is 
noise, activity, other things to watch, girls, boys... On the computer the screen is tightly 
organized, you know immediately whether you passed the module, you have headphones 
[to focus attention and block noise], the screen is colorful and active. One student said, ‘I 
could never get it in the classroom. The teacher doesn’t have time to focus on me. The 
computer is teaching me alone, not 25 other kids.’ We’ve worked with students 13 to 18 
years old. It works with all those ages. 

"It’s obvious as soon as they sit down and grab the mouse: kids like it. Because they like 
manipulating the computer, they learn better than they would otherwise. They would sit 
there all day if you let them. The software can handle repeated reviews of information; 
that is tough to do in the classroom, where the teacher has to keep the whole class 
moving. Test preparation is particularly well suited to computerized instruction because 
there is a clearly defined set of objectives. 

“We use PLATO because it’s the best thing on the market right now. We’re not selling 
software — we’re selling skill development. PLATO is the best tool we’ve found for that 
purpose. That’s why we use it in our programs. 

"For a state that prides itself on high tech leadership, Massachusetts makes far too little 
use of computers in public education. We’re working hard to educate our legislature on 
the usefulness of computers in the standards-based education reform environment. They 
need to see data to show what computers can do in the classroom. Legislators are very 
willing to entertain new ideas and to fund them if the data support the effort. They are 
sometimes even ahead of professional educators when it comes to trying new approaches. 



"It’s easy to grasp the usefulness of computerized instruction if you understand the 
structure and scoring of the MCAS. The 2000 10 th grade math test had 32 multiple choice 
questions, each worth one point; four short-answer questions worth one point each; and 
six open-response questions worth four points each. All these questions added up to 60 
points. The raw point scores were converted into scaled scores which ran from 200 up to 
280. A raw score of 21 points translated into the passing scaled score of 220. Those 21 
points could have been obtained by answering 21 of the 32 multiple-choice questions 
correctly. 

“In that year (2000) 28,000 10 th graders got less than 21 points. Of those 28,000, almost 
8000 scored between 16 and 20; another 10,000 scored between 1 1 and 15. A few more 
right answers on multiple choice questions would have had a dramatic effect on the 
failure rate. Five more right answers would have moved 8000 youngsters over the passing 
line; ten more right answers would have cleared an additional 10,000. In percentage 
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terms, ten more right answers on multiple choice questions would have cut the state’s 
failure rate from 45% to 16%. Our program can produce those ten right answers. 

“That’s what we’re saying to the legislature and to the public schools. We know what 
skills the MCAS tests: we have four years of actual test questions to work from. We also 
know how to build a test prep course in PLATO to teach the skills students need in order 
to achieve those 21 right answers. It’s not some kind of trick test-prep drill: it’s old- 
fashioned skill development using a new teaching tool. 

“Passing the MCAS is an attainable goal. There’s a way to do it, and we can teach people 
that way. Anyone can learn to use PLATO. It doesn’t have to displace or disrupt current 
practice; it fits right in and supplements anything that’s being done in the classroom. It’s 
really a multi-purpose tool, the Swiss Army knife of software.” 

Madison Park Headmaster Charles E. McAfee summed up his conclusions as follows: 
“Our staff who were involved in the summer program observed several factors that help 
to explain the unprecedented success of the computerized instruction. 

• Instruction can be designed individually for each student, so that the 
individual gets exactly what he or she needs. 

• Instruction is self-paced, activated by clicking on a prompt; this prevents the 
student from being left behind by a lesson moving too fast, or being bored by 
a lesson moving too slow. Each student proceeds at his or her own pace. 

• The interactive presentation of math captures and holds students’ attention. 
Color and dynamic graphics on the screen, audio tracks, interactive devices 
such as drag-and-drop give the feel of television and video games. Teenagers 
are children of the media age; computers are their element. They respond to 
computerized math lessons as they would to a video game. Their focus is 
longer and their concentration more intense than in a classroom. Focusing on 
the screen eliminates environmental distractions, so that the proportion of time 
on task in the class period is higher. 

• The quality of the content is high. JFY uses PLATO, an established integrated 
learning system that has been on the market for more than 30 years. PLATO 
is one of the most respected software products on the market. The company 
has a good reputation for service and for product improvement, issuing 
frequent updates to keep its software in the forefront of innovation in a highly 
competitive field. 

• JFY provides ongoing support to teachers in the classroom. High quality 
software by itself does not produce learning gains. Teachers must be trained 
and given support in the classroom. JFY provides this training and support. 
JFY also provides technical backup to fix hardware and network problems. 

• The PLATO software provides frequent and accurate assessments. Student 
progress can be tracked on a daily basis. If corrective action is needed, it can 
be taken quickly, without loosing precious instructional time. 
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• PLATO provides detailed records of student activity. Time on task and 
progress through instructional units are recorded in the management system. 
The teacher can easily determine precisely what each student has done. 

• PLATO’S curriculum modules can be aligned to multiple learning objectives. 
The flexible structure of PLATO allows lessons to be sequenced by the 
teacher, so that alignments can be made to the MCAS, SAT, Stanford 9 or any 
set of objectives.” 

PLATO Learner Survey Results 



Fifty-four students answered a PLATO student questionnaire. Whether these respondents 
were representative of the entire group in their computer skills and reading skills, or 
whether they spoke English as their first language, could not be determined. 

Nevertheless, this is a big enough response rate (approximately 50%) to give some 
insight into the learning experience that participating students had with PLATO. 

System Usability. Most of the students felt very comfortable in using the PLATO 
system. Nine out ten (90%) said that they had control over lessons, starting and stopping 
when they wanted to do so. Almost as many (88%) said that they could get to their 
lessons easily and that they could work at their own pace. Eight out of ten students 
(83%) rated computers as easy to use. A majority of the respondents (85%) were able to 
sign on to the program without problems; one of ten (9%) disagreed and a few (6%) were 
neutral. This opinion was confirmed by asking a similar affective question from a 
negative direction (a good survey technique): 83% of the students disagreed with the 
statement, “The computer makes me nervous.” 10% agreed, and 6% were neutral. 

Most students said that PLATO was interactive, which is viewed as a positive attribute of 
the system. Eight out of ten student respondents (83%) agreed that, “The computer lets 
me answer often and not mainly just watch.” Three out of four respondents (74%) said 
that the computer gives help when they need it, and that they would recommend learning 
from a computer (72%). (Items 4, 2, 8, 13, 7, 15) 

Perceived student Outcomes. The majority of the student respondents said that 
using computers was a positive experience for them personally. Approximately two- 
thirds (65%) said that working on computers made them feel good about themselves 
(17% disagree, 22% neutral). Those who disagreed or were neutral may have been 
saying that computers did not affect their feelings about themselves either way. When 
worded in the negative, “When I give a wrong answer on the computer, I feel bad,” 
nearly the same number of respondents said that they disagreed (58%), thereby saying 
they felt fine about their experience, regardless of the short-term outcomes. Even when 
students did feel bad about poor performance outcomes, their response may indicate that 
they cared about what they were doing. 

When asked about how much they could understand from the computer, a few students 
gave somewhat lower ratings; but again, the majority said that they used the system 
easily. This is noteworthy, considering the academic level of this summer school group. 
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Despite their low performance on tests, they could read and participate in the PLATO 
lessons with comprehension. Two-thirds of the respondents (67%) said that they “could 
understand what the computer teaches . . .without help from an instructor.” Nine students 
(17%) disagreed with this statement and eight (15%) were neutral. 

Somewhat fewer students (49%) agreed with the statement, “The lessons on the computer 
are designed for people like me,” but the remainder were more likely to be neutral (34%) 
than negative (17%). These responses do indicate many of these students had some 
concerns about the instruction, in spite of their ease in using the computer and the system. 
To the statement “I feel I’m studying what I need to on the computer” only one-third of 
the students agreed and most of the remainder disagreed (58%; 9% neutral). 

Student Open-Ended Questions. Students were asked to write their opinions on 
each of four questions. Generally, students were very positive about the program. They 
liked the individual learning experience with clear instruction, adequate practice, and 
immediate feedback on learning. They wanted updated computers that would run the 
program faster. Several students recommended using the computers in the regular school 
instruction. 

1. What do you like best about learning from the computer? 

• It helps me learn better. I think if we had this program last year, a lot of students 
would not have failed the test. 

• I like it because it’s just me and the computer. It’s like I’m the only person in the 

class. , 

• I like it because PLATO allows you to work at your own pace. 

• It allows me to work quietly on my own. 

• You can take as long as you want. 

• Learning from the computer is easier than in the classroom. It takes you step by 
step at your own pace. 

• It helps you solve equations by giving you step by step instructions. 

• When you leam something, you have practice to keep it straight in your mind. 

• When you get a question wrong, it goes back till you get it right. 

• It gives you more help. 

• It makes me feel good when I get something right! 

• It gives examples of how to do the problem. 

• I like it because the problems are like the ones I leam in class. 

• I get to review all the things that I forgot in class. 

• It gives you the answers after a few times. 

• I like the math! 

• It tells me the work I must do. 

2. What do you like least about the computer? 

• Sometimes the computer makes you put in the exact way the answer was 
programmed and that might not be the way you learned it in class. 

• Sometimes you need the instructor to explain the question. 
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• The computer takes too long to get to the next problem. 

• The comments they make and that it is very slow. 

• A little tired to use. 

• I can ’t think of anything. 

• Nothing. 

3. How would you change the computer lessons or the way you use them? 

• I would not make the lessons so long. 

• I would add in more word problems. 

• Make easy questions. 

• I would change it by mixing it up with different things. 

• I would have games on it all day long. 

• I would make the computer go faster. 

• I would have someone monitor the students. 

• The truth is I would not change the computer lessons. 

• Everything is fine. 

4. What other suggestions do you have to improve any part of this course? 

• The suggestion I have is for everyone to have this program for next year. 

• Each room should have computers so you can use them every day. 

• I would add a list of websites. 

• Do more with moving objects, like when you get a question right. 

• Update the computers, they’re too slow! 

• Concentrate hard and pay attention to every question. 

• Keep doing what you’re doing. . .this program is a great learning tool! 

• Just keep doing what you’re doing. 

• Everything is great. 

• No suggestions. 
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PLATO Learner Ratings (n=54) 



4.1 can start and stop a lesson 
whenever I want. 



8. I can work at my own pace on 
the computer. 



2. Getting to my lessons is easy. 



3. The computer is easy to use. 



1 . 1 am able to sign on to the 
computer without problems. 

5. The computer lets me answer 
often and not mainly just watch. 

15. I recommend learning from 
the computer. 

7. The computer gives me help 
when I need it. 

6. I usually can understand what 
the computer teaches me, without 
help from my instructor. 

14. Working on the computer 
makes me feel good about myself. 

1 2. I would like more time to study 
on the computer. 



10. The lessons on the computer 
are designed for people like me. 

1 1 . When I give a wrong answer 
on the computer, I feel bad. 

9. 1 feel I’m studying what I need 
to on the computer. 

13. The computer makes me 
nervous. 
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Mean PLATO Learner Ratings (n=54) 
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Conclusions & Discussion 



Learners enrolled in the summer math program involving both BPS teacher-led class 
instruction and PLATO math modules increased from an average pre-test of 32 percent 
right on the BPS Math test to a post-test of 5 1 percent correct. This meant that 42 percent 
of the math learners in the summer program were able to pass the math graduation 
requirement for the 9th grade. 

Learners enrolled in the summer reading program involving both BPS teacher-led class 
instruction and PLATO reading modules increased from an average pre-test SRI lexile 
score of 704 to an average post-test score of 970. This was a very large increase and 
meant that 70 percent of the reading learners were able to pass the reading graduation 
requirement for the 9th grade. 

The evaluation also showed that greater learner mastery of PLATO modules for math, 
and time on task for reading, was moderately positively correlated with higher post-test 
scores and pre-post gains. This suggests that the benefits from PLATO courses increase 
when the student masters more modules or spends more time studying PLATO content. 

These findings are consistent with the staff observations. They described how students 
were assigned PLATO modules targeted to the content of the tests. The software 
provided instruction that was tightly focused on instructional goals and monitored by 
interactive questions and answers and occasional tests. Students were very engaged in 
learning on the computer; staff said that learners were less distracted working on the 
computer than in a group instruction setting. Students were allowed to learn at their own 
pace and were tested to be sure they had learned the material before going on. Students 
received personal assistance in the computer labs to keep them on track; the presence of 
support staff in the lab was very important. 

Also of note are staff perceptions that students who were marginally in the program — 
those who had almost passed the test or tests — did best on the PLATO system. They said 
that those with lowest test scores also struggled with PLATO. However, except for those 
with serious language deficits, even the low-scoring students learned in this computer 
setting. 

Overall, student responses were extremely positive about PLATO. A strong majority felt 
comfortable getting around in the system and participating in the instruction. Most of 
them enjoyed their computer work; a few were neutral about it. Very few had negative 
feelings about themselves for missing answers. There was some concern, however, about 
the content of lessons and whether some of the PLATO work was what students thought 
they needed. As noted earlier, it is clear that some respondents may have had language 
difficulty or a disability that limited their comprehension of PLATO instruction. 
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Appendix: Supplementary Tables 



Table A 1 . Number of PLATO Math Modules Mastered 



# 

Modules 


Frequency 


Percent 


Cumulative 

Percent 


1 


3 


3.9 


3.9 


2 


8 


10.4 


14.3 


3 


7 


9.1 


23.4 


4 


10 


13.0 


36.4 


5 


7 


9.1 


45.5 


6 


6 


7.8 


53.2 


7 


6 


7.8 


61.0 


9 


4 


5.2 


66.2 


10 


5 


6.5 


72.7 


11 


2 


2.6 


75.3 


12 


2 


2.6 


77.9 


13 


1 


1.3 


79.2 


14 


1 


1.3 


80.5 


15 


1 


1.3 


81.8 


16 


2 


2.6 


84.4 


17 


1 


1.3 


85.7 


18 


1 


1.3 


87.0 


19 


3 


3.9 


90.9 


20 


2 


2.6 


93.5 


21 


1 


1.3 


94.8 


22 


1 


1.3 


96.1 


23 


1 


1.3 


97.4 


26 


1 


1.3 


98.7 


48 


1 


1.3 


100.0 


Total 


77 


100.0 





Table A2. Grouped Math Hours on PLATO 



Hours 


Frequency 


Percent 


Cumulative 

Percent 


0 to 3 hrs 


2 


2.6 


2.6 


3 to 5 


3 


3.9 


6.5 


5 to 7 


10 


13.0 


19.5 


7 to 9 


16 


20.8 


40.3 


9 to 11 


12 


15.6 


55.8 


11 to 13 


8 


10.4 


66.2 


13 to 15 


12 


15.6 


81.8 


15 to 17 


7 


9.1 


90.9 


17 and up 


7 


9.1 


100.0 


Total 


77 


100.0 
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Table A3. Number of PLATO Reading Module s Mastered 



# 

Modules 


Frequency 


Percent 


Cumulative 

Percent 


3 


3 


6.4 


6.4 


4 


2 


4.3 


10.6 


5 


2 


4.3 


14.9 


6 


6 


12.8 


27.7 


7 


6 


12.8 


40.4 


8 


24 


51.1 


91.5 


13 


1 


2.1 


93.6 


15 


1 


2.1 


95.7 


16 


1 


2.1 


97.9 


17 


1 


2.1 


100.0 


Total 


47 


100.0 





Table A4. Grouped Reading Hours on PLATO 



Hours 


Frequency 


Percent 


Cumulative 

Percent 


0 to 3 hrs 


16 


34.0 


34.0 


3 to 5 hrs 


13 


27.7 


61.7 


5 to 7 hrs 


9 


19.1 


80.9 


7 to 9 hrs 


8 


17.0 


97.9 


9 to 1 1 hrs 


1 


2.1 


100.0 


Total 


47 


100.0 
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